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Many programs for individualization of instruction 
are based on a curriculum structure involving a careful sequencing of 
instructional objectives and larger units of learning content. A 
methodology that may be useful in the formative evaluation of the 
sequential and structural properties among predefined curriculum 
units and specific instructional objectives was investigated. The 
three distinct research populations consisted of the student 
enrollment in three schools employing the Individually Prescribed 
Instruction (IPI) system for elementary school mathematics. To 
examine these structural relationships, as they are found in 
different content areas and levels, student performance on unit 
placement tests and pretests in thirty IPI mathematics units 
consisting of 173 specific objecti-ves was investigated through 
scalogram arid simplex analyses. The usefulness of these procedures irt 
providing suggestions for the revision and refinement of curriculum 
structures was demonstrated. Alternative structures and sequences 
suggested by this evaluation progra may be helpful in providing 
guidance for curriculum a.pd testing specialists in the design of 
future curriculums in IPI mathematics. (Author/PR) 
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Among the programs for the individualization of instruction in 
current use, many are based, on a curriculum structure involving a careful 

i* 

sequencing of specific instruction objectives as \^ell as larger units 
of learning content, The formative evaluation of such systems of individ- 
ualization should include the study of the extent to which actual pupil? 

performance supports the/va^Lidity and meaningfulness 'of the hypothesized 

/ \ 

y , * y* \ * 

learning sequences atfd struc Cures. Gagne (1967), Rcsnick (1968), and 

• ,v ^ 

others'h'ave proposed methods for generating and assessing such structures. 
Among procedures which have been proposed, but investigated only to a 
limited extent, are ' Guttman 1 s procedures of scalogram analysis (Guttman, 1950) 
and simplex analysis (Guttman 1954) . The purpose of the* present study v?as 

• i 

to investigate the usefulness of these two procedures for various specific 
steps in the formative evaluation of an individualized mathematics program, 
the Individually Prescribed Instruction system. 



Scalogram Analysis 

Scalogram analysis provides a technique for testing the existence 
of a single quantitative variable, referred to as the scale variable, in a 
predefined universe of content. From a person's score on a measure of 
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this scale variable one knows precisely the person* s responses to each 
of the items or objectives constituting the scale*, Thus, if a pass-fail 
response pattern of a s^tud-e-nt- sample for a gfoup of objectives forms a 
scale, then mastery of a given objective will imply mastery of all sub- 
ordinate objectives and non-mastery of a given objective will imply non- 
mastery of all supraordinate objectives. . 

\ ** % 

The definition of a perfect scale, as used in this study, is one 

in which a person's pass-fail performance with respect to mastery of each 

y 

group of objectives may be reproduced from his scale score. The scale score 
is found by counting the total number of objectives which have been mas- 
tered. Thus, scales derived from sealogram analysis have^the property that 

—'-A 

responses to individual objectives are reproducible from tpe scale scores. 



lining 



A two-step procedure may be used for determining the existence of a scale: 

/' • 

(1) rank the objectives from the' highest to the y lowest in terms of the per- 
centage of students indicating mastery, and (2) rank the people from the 
highest to lowest in terms cf total score; i.e., total number of objectives 
mastered. The resulting pattern of a perfect scale will be triangular 
in, shape when only mastery responses are recorded. An example of a perfect 
« scale is shown in Figure 1. / 

Perfect scales, however, are rarely found in practice. The measure 
for the degree of approximation to a perfect scale is defined as tire 

* 

i . r 

coefficient of reproducibility* This coefficient provides an index of 
how well a person's response pattern can be predicted from his scale score. 
The coefficient is computed by counting the total number of errors, or 
discrepancies from a perfect scale pattern, finding the ratio of the errors 
to the total number of possible responses and subtracting the ratio from 
unity. , |( 
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V Lingoes (1963), in a procedure known as multiple s'calogram analysis, 
has dev<hre-{ie<i\ a method for analyzing the response pattern of a set j)f 



dichotomous variables for the purpose of searching out optimal sCalable 

.. ^ ’ r o* 

subsets within larger sets bf data. Rather than reject -the scalogram 



hypothesis for the total set of variables in cases wherb a single sc/^e/^* 
is not obtained, the method seeks out smaller sets of va^ ( ia^Les which 
may, indeed, form scales with high coefficients of reprooucibility • The mul- 
tipis scalogram procedure has been programmed and was used in this study, A 
description and explanation of the program has been reported by Weisberg (1966) . 

J Simplex Analysis 

A second method of order analysis which' appears to be capable of 
providing answers to questions regarding order and structure among behav- 
ior ajrhb jec tivve.s is "simplex analysis. n ^~G4^ttman describes this theoretical 
position as "A New Approach to Factor Analysis: Xv The--Rade>. , n (Guttman, 195^ ) 
where the term "radex" indicates a r a‘di a i o n of compjlexity^ The 
simplex is one element of the more genera^. rade,x,) and indicates a simple 



order of complexity among .variables of the sjame kind. ^Wneycircumplex is the 
^-..^rB^naining element of the radex and indicate^a circular order ataong 

dif f'e^nt ^ki nds of variables when they are ali^ of the same magnitude of 
complexity. 'The ana lyiTc^i-^app r o a c h^af "^llrvpJL^x analysis, t^jerefore, was 
developed in order to investigate the hypothesis of complexity hierarchy* o 
among observed quantitative variables by accounting for the positive 
correlations among the variables. 

It appears that simplex analysis is ideally designed for investi- 
gating order among instructional units and would be a preferred method 
for studying hypothesized linear sequences when continuous quantitative 
achievement data were available for. each content unit. The simplex 
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hypothesis is that the corftent units may be arranged in a hierarchical order 
of complexity such that the inter correlation matrix ofj the Pearson prpduct- 
moment coefJjLcieixts are accounted for by a single complexity factor. If this., 

tesis is verified by student response data, then one has *a justifiable 

VdL ,> 

basis for considering the existence of a ItjtprarchicaT or prerequisite relation- 
ship among they variables . This analysis may constitute a curriculum design 
validation procedure when the hierarchical relationship among instructional 
units has been pVeviousi^ hypothesized. It is suggested here that simplex 
analysis is better/suited for investigating large segments of a curriculum 

such as units of an arithmetic curriculum in such topics as numeration, 

\ / * . 1 
N ‘ 

N | f 

addition, etc., whereas scalogram* analysis is better suited for analyzing 

ihe dichotomized pass-fail data perived from performance on individual in- f 

i , \ ( 
structional objectives. \ ^ 

Kaiser's (1962)"'method for scaling the variables ^af-^a simplex v;as 

used in this study. The procedure yields^ a least squares solution for 

V\ 9 

ordering the variables and provides^ a measure of the goodness of fit for 
the empirical data. 



The Setting for the Study 




To investigate structure and sequence within a curriculum in terms 
of the learner T s performance, it is necessary to work within a setting where 
the units of ' the curriculum have been, defined in measurable human behavior. 
These units may vary from a single instructional objective involving a specific 
skill to an entire content area consisting of many objectives. Since the mathe 
matics curriculum of the Individually Prescribed Instruction (IPI) program 
developed at the Learning Research and Development Center, University of xJ 

Pittsburgh, is based on units defined in this way, it has been used to demon-' 
strate the formative evaluation methodology proposed in this study. 

5 
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In the organizational plan" of the present mathematics curriculum 
of the IPI Project, an effort \<ras made to- design objectives with an order 
of complexity relating to two^ dimensions, (l) content area and (2) level 
c^f achievement^-" In general, tlie > texpected^ route by which a student would 



I^rpglTTiss- th jU h the curriculum is: (1) within a given level the pro- 

\ 

gressipn wouTd be from the least complex area to the most complex area, 

and (2) all obj ectives :gn a given level would be mastered before advancing 

to the next higher level) progressing from the least complex level to the 

most complex level. A curriculum structure such as^this implies an order 

of complexity both across predef in^c^^eas (numeration, place value, 

addition, subtraction, multiplication, division, etc.) 

defined levels (B, C, t), E, G, etc.). The numbers of objectives in 

each area and level are shown in Table 1. Thus, a student will typically 

work through the series of content areas at one level before moving on to 

the next. The work in a given content area at ^Hgiveti level, such as 

Numeration, B-level, is. identified as a unit and is defined on the basis 

of a limited^ number of specific instructional objectives. In this study 

an abbreviated notation Xtfill be used to refer to a particular objective 

within the: curriculum. Thus, NB1 will refer to the Numeration area, B- 

level, /nd to the first objective listed for that unit. It is assumed 

r ""V 

that the objectives are arranged in an order of complexity within eacti^- 
unit. Thus, in addition to the study of the hierarchical relationship 
among the broadly defined units as above, this curriculum* organization 

also provides for the study of the structure and sequence among specific 

% 

instructional objectives which comprise these units. Empirical evidence 
concerning both of these types of hierarchies is important in evaluating 
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the validity of euAriculum structure as a guide to pupil placement 

\ 

arid the prescription of pupil learning activities. 

for the diagnosis of individual nced^^cevcral criterion- 

\ -/* ’ - ; 

referenced in^truiiients have bo.cn developed for the placement and 

monitoring of students vj^thin the IPX 

nieiKS, placement tests and pre tests, hbve been used to obtain data for 

this study arid are described as follow.^ 



r 



urriculum, Two -of these instru- 



Placement Tost 

Placement tests are administered tit. the beginning of each school 
year, or, for a new pupil, when he eater^sichool, % These tests are broad 
in scope as t Vi fey are intended to provide a general profile of individual 
pupil achievement over many units of work, The placement tests a r - content 
referenced in that each item on eye ry test is coded to one^articular 
objective in the curriculum. Since placement tests must be of minimum 
length while providing a maximum ^OjJ^Tlformation , not every objective in 

V ' ■ ' c " 

each unit is testedN Generally the most important, or most characteristic 
objectives in k unit/are tested. The basic information proyided by the 
placement tdsth' is "a measure of the highest level of ^\pyery that the pupil 
has attained in each topic area (Gox and Boston, 196Z) 



Unit Pro 


test 




V r ' 



A^pretcst is administered when a pupil is about to begin a unit. 
Since the items are designed to measure the achievement of the skills 
specified in the cur/ioulum for that unit, each objective is represented 

and the pupil receives score on each skill or objective. If the pupil 

[ 

achieves mastery (85 percent) in all skills on the. pretest, he does not 









\ 



work in that unit, but is given the pretest for the next unit in his 
instructional sequence. If mastery is not achieved on a particular skill, 
“tlys pupil is prescribed work within that skill (Cox and Boston, 1967) , 
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Purposes of the Study . ’ *■ 

Cpecifically the study examined the following applications of 
scalogram and simplex analysis procedures,*, 

'■ 1. The study of th£ extent to which the hypothesized 

ordered sequences of specific objectives are sup- 
ported by pupil test performance. 

- 2, The study of the extent to which the hypothesized 

order of various mathematics topics (numeration, 
addition, subtraction, etc.) at a given level in 
the curriculum is supported by pupil test, performance. 

3, The study of the extent to which the hypothesized 
order of unit levels within a given topic area 
is supported by pupil test performance, 

4. A comparison of results obtained by scalogram analysis 
and by simplex analysis when used for the foregoing 
analyses . 






Methods 
* * 

Guttman'^ scalogram analysis and simplex analysis were applied 
to test results obtained fromtfth^ administration of IPI Test Batteries 
to students in three schools using Individually Prescribed Instruction, 

The application involved_tne computation of reproducibility coefficients 

\ 

and of permutat-iprt indices for comparing , differences between hypothesized 

and empirically derived s calcs % '* 

/ 
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Data Sources 






, Data analyzed included 1PI Placement Test and Unit Prete f £tf~Tcsults 
* ’ for thirty units of mathematics consisting of a total of 173 specific 
objectives. Three samples of students' were used, consisting of the 
enrollment in three schools employing the IPI mathematics program. 



The results, 
four major purposes. 




Results 

s study can be summarized in terms of the 

r\ * 
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The Study of Hypo tliegi zed' ’Ord e red Seq u ences of Specific Ob jective^ f 

Nineteen homogeneous groupings of objectives were hypothej6ized^~\^y/ 



V 

hiH% 



withi^Q»:hree samples tested by< a placement test program at one of 7 t^he 



three demonstration schools. 



c " V 



Numeration and Place Value . Eight hypothesized scales \ji£hin 

the content areas of Numeration and Place Value are shown in Tables 2, 

/ 

/ 

3,- and 4. The results of a m'^tiple scalogram analysis including the 

suggested optimum empirical scale, the reproducibility coefficient, 

/ s \ . 

and the percentage of students indicating mastery of each objective 

is stiow£. A minimum reproducibility coefficient of .80 has been used 

v ^ — 

throughout this study as one of the criteria for the existence of a 
scale. A measure of the difference between the hypothesized scale and 
the empirical scale has been operationally defined as . the ^permutation 

tf 

index and is computed by^counting the number of permutation inversions 

/ # . O ■ 

required to make the hypothesized and empirical scales identical. The 



total number of inversions in a permutation is, by definition, found by 
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counting the number of smaller integers following each ^Integer of the 
permutation. Thus 614325 has eight inversions since 6 is followed by 
1, 4, 3, 2, and 5; 4 is followed by 3 and 2; 3 is followed by 2. Thus, 
a permutation index of zero indicates the two scales are identical, and 



the larger the index, the greater the scale difference. 



j 




The homogeneous grouping of objectives in the area of "Expanded 
Notation - Integer Numbers" is shpvn in Table 2, The hypothesized scale 
for the nine objectives in this scale is not verified by the scalogram 
results, The permutation index of 13 indicates a high degree pf difference 
between the hypothesized and empirical scales. Even though a sjingle scale 
was derived from the multiple scalogram analysis, the difference in the 
order of the objectives between this empirically derived scale and the 
hypothesized scale suggests that either the hypothesized scale is in error 
or the test items keyed to the objectives are not measuring what they were 
designed to measure. Thus, depending upon the assumption which the 
investigator is willing to make, the results of a scalogram analysis which 
does not verify the -original hypothesis may be interpreted as showing that 
either (1) the hypothesized scale is in error or (2) the test items measuring 
mastery of the objectives are invalid in the sense that they are net 
measuring what they were intended to measure,, There is no reason why 
- one of these assumptions * s bas^ally superior to the other. In some 
cases, the indicated position of a particular objective may be the result 

of improperly written or invalid test items. If, however, it is agreed 

<» / 

that the test items can be assumed to be valid, then this information pro- 
^ vides'-'guidance for re-thinking the logical relationship of the subject 
matter as defined by the instructional objectives. This way of interpreting 
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scalogram results emphas. . the importance of jointly seeking solutions 
to the logical signifies: ■? of scales by people interested in both 

curriculum structure design and .test design. 

The homogeneous grouping of objectives in the area of "counting" 

< 

is shown in Table 3, The hypothesized scale for the seven objectives 

in this group is verified by the multiple scalogram results. A single 

scale with the same hypothesized order resulted from the analysis. A 

high degree of consistency in the student responses to the test items is 

indicated by the single scale* and since the hypothesized order is 

verified within the homogeneous group, we may interpret 'these results.. 

as a validation of the test items for measuring what they were designed 

to measure. The reproducibility coefficient of .946 may be interpreted 

as a validation of the hypothesized order among the objectives and of 
\ ' 

th(B test items which measured mastery' of these objectives. 

The remaining six hypothesized groupings of homogeneous objectives 

in numeration and place value are shown in Table 4. Only one of these 

/ 

groups, "Expanded Notation — 'Decimal Numbers," requires a reexamination. 

The hypothesized 'scales in all remaining groups were verified by the 
multiple scalogram analysis. 

Addition and Subtracti on, Five hypothesized scales within the 
content area combination of Addition and Subtraction are shown in Tables 
5, 6, and 7. The homogeneous grouping of objectives in the area of 
"Integer Addition — Facts and Algorithm" is shown in Table 5. A single 
empirical scale is established by the analysis. The permutation index 
of 2, hov/ever, reflects a reversal of the scale positions of AF1 and 
AE6, and AD8 and AD6, respectively. This analysis suggests that the test 

items and sequence of these four objectives should be reexamined. 

O 1 
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The homogeneous grouping of objectives in the area of "Integer 
Subtraction — Facts and Algorithm” is shown in Table 6. The hypothesized 
scale for these five objectives is verified by the analysis except for 

m 

the reversadT^of" the scale positions of SCI and SC3. The test items and 
sequence of these two objectives should therefore be reexamined. 

The remaining three hypothesized groupings of homogenieoiSS^b j ectives 
in Addition and Subtraction are shown in Table 7. The empirical scales 
resulting from these analyses suggest that the test items and sequence of 
the objectives in each group should be reexamined with, the exception of 
SF1 in the "Addition-Subtraction Decimal Number" group. 

Multiplication and Division * Six hypothesized scales within the 

content area combination of Multiplication and Division are shown in 

* 

Tables 8, 9, and 10. The homogeneous grouping of objectives in the area 
of "Multiplication Algorithm — Integer Numbers" is shown in Table 8. A 
single empirical scale is established by the analysis for these five 
objectives. A reversal of objectives MF5 and MF10, however, indicates 
that the sequence of these objectives and test items should be reexamined. 

The homogeneous grouping of objectives in the area of "Division — 
Concepts and Facts" is shown in Table 9. The order of three of the four 
objectives in this scale is not verified by the analysis. The sequence of 

r f 

objectives and test items corresponding to DD2, DD5, and DD7 should be 
reexamined, based on these results. 

The remaining four hypothesized groupings of homogeneous objectives 
in Multiplication and Division are shown in Table 10. Only one empirical 

scale, "Multiplication — Concept and Facts," suggests an order different 

\ 

from the hypothesized scale. In this case, the sequence of the objectives 
and test items for MD3 and MD4 should be reexamined. 
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Due to the extremely low mastery percentages for the remaining 
three scales, any judgement on the test items corresponding to these 
objectives should be withheld. 



The Study of Hypothesized Order Among Content Areas 

As stated earlier, the organizational plan of the IPI Mathematics 
Curriculum implied an order of complexity among the units relating to two 
dimensions, (1) content area and (2) content level. The results of the 
study of these complexity orders will now be presented. 

The placement test results from a demonstration scho'ol”l^&r‘eh-.~' 

analyzed for the purpose of investigating the complexity order among 
the content areas within a given level. Subsets of all students within 
the school who received unit scores for each of the six content areas 
were determined for levels D, E, F, and G. Analyses were conducted only 



for levels D and E, however, because of inadequate sample sizes for 



/ 



/ 

/ 



levels F and G. 

/ 

Level D Complexity Study , Each student in the sample group of 
level D received a percentage score on the unit subtests corresponding 
to the areas of ..numeration, place value, addition, subtraction, multiplica- 
tion,' and divis.ion. The tinier correlations among the unit scores for these 

V. 

six areas are shown in Table 11 where the topic or content areas are listed 

fv/ • 

in the order used in the IPI curriculum Mi. 

Table 12 shows the re-orderii>g/of! 'jtopic areas which resulted from 
application of ^implex analysis. F^r' this optimal ordering a "Q squared" 
value is computed to indicate ^h^extent to which this ordering can be 
explained by a single complexly factor. For the optimal ordering of 
the content areas as shown in Table 12, th^,-Q-'squared value is 0.971. 
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This indicates a high degree of success in explaining these correlations 

* 

in terms of a single complexity factor among tlio content areas, when the 
areas are defined in terms of the objectives included on the unit place- 
ment tests. This analysis suggests that a single complexity continuum, 
from least to most, exists ante ng v the content areas of. level D in the 
following order: addition, /multiplication, division, subtraction, numer- 

t 

ation, and place value. Proper interpretation of these results depend 
'upon the assumptions that (1) the objectives included on the placement 
tests adequately represent the units and that (2) the test items repre- 
senting the objectives are valid. Under these assumptions the results 
of level D simplex analysis would cast doubt upon the hypothesized com- 
plexity continuum, from least to most, of numeration, place value, 
addition, subtraction, multiplication,, and division. The results might 
be interpreted as suggesting the preferred or optimum progression among 
units within level D of tlifs curriculum. At least these results would 
tend to discourage a rigid, preconceived notion of how a student should 
progress through level D*of this curriculum. 

Level E Complexity Study . Each student in the sample group of 
level E received a percentage score on the unit subtests corresponding 
to the areas of numeration, place value, addition, subtraction, multi- 
plication, and division. The inter correlations among the unit scores 
for these six areas are shown in Table 13. 

The output of the simplex analysis on this matrix, using Kaiser ! s 
algorithm for scaling the variables, is shown in Table 14. 

For the optimal ordering of the content areas as shown in 
Table 14, the Q squared value is 0.965. This represents a high degree 
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of ’‘goodness of fit" between the predicted matrix based on the single- 
complexity factor loadings, and the original matrix. Therefore, thesi. 
correlations can be explained quite successfully in terms of the 
existence of a single complexity factor among the content areas, when 
the areas are defined in terms of the objectives included on the unit 
placement tests. This analysis suggests that a single complexity continuum 
from least to most, exists among the content areas of level E in the 
following order: subtraction, addition, division , multiplication, place 

value, and numeration. 

Under the assumptions of valid test items and the validity of 
the unit representativeness of the placement tests* the results of level 
E simplex analysis would cast doubt upon the hypothesized complexity 
continuum, from least to most, of numeration, place value, addition, 
subtraction., multiplicatio n and division. These results represent a 
rejection of this curriculum. The suggested order might be interpreted 
as an alternate hypothesis regarding the preferred route for students to 
progress through this level. 

The Study of Hypothesized Order Among Content Levels 

The placement test results from a third demonstration school were 
reviewed for the purpose of investigating the complexity order among the 
content levels within a given area. One sample group received unit scores 
for each of the six levels, B, C, D, E, F, and G in the content areas of 
numeration and place value. A second sample group received unit scores 
for each of the five fevels, C, D, E, F, and G in the content areas of 
addition and subtraction. A third sample group received unit scores for 
the four levels D, E, F, and G in the qjontent areas of multiplication 
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and division. The results of the complicity study among levels will 
now be discussed separately for the content areas of numeration and place 
value . 

Nu meration Co mplexity Study , Each student in the sample group 
of numeration received a percentage score on the unit subtests corres- 
ponding to levels B, C, D, E, F, and G. The intercorrelations among the 
\ 

unit scores for these six levels are shown, in Table 15. 

' An analysis was performed on this intercorrelation matrix by 
using the computer program employing Kaiser’s algorithm' for scaling the 
variables of a Guttiman simplex. The output of the simplex program is 
shown in Table 16. 

For the optimal ordering of the levels as shown in Table 16, the 
Q squared value is 0.992. Therefore, these correlations can be explained 
quite successfully in terms of the existence of a single complexity factor 
among the content levels as defined by the unit placement tests. The com- 
plexity order, from least to most, of the levels within the content area of 
numeration as suggested by this analysis is C B.D E F G, 

Place Value Comple x ity Study . The same sample group used in the 
numeration complexity study also received subtest scores on the unit place- 
ment tests in six levels of place value for the purpose of investigating 
their complexity order. The intercorrelations among the unit scores for 
these levels are shown in Table 17 where the order of levels is the alpha- 
betical order used in the IPI curriculum. The optimal ordering of the levels 

f . 

as determined by simplex analysis is shown in Table 18. The rather com- 
plete lack of agreement between these two orderings suggests that in the 
topic area of place value the units of study, as currently organized, are 
not arranged in the proper order of increasing complexity. The Q-squared value 

O 
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of 0.957, computed for this ordering, indicates the existence of a single 
complexity factor for the correlations as reported in Table 18. 

A Co mpa rison of Results Oh ta iued by Scal o gram A na 1 y si s and by 
S imp 1 ex Analysis 

/ 

Tab Ip. s 19, 20, 21, 22, 23, and 24 provide a comparison .of the 
results of simplex analysis and scalogram analysis for six different 
content ureas^ These tables also permit the comparison of the two ern- 

■>iiw 

pirically derived scales with the original hypothesized scale since the 
latter is represented by the alphabetical order (starting from the bottom 
of the table) of the letters assigned to the curriculum levels. 

Table 19 shows that the scalogram analysis supported the hypothe- 
sized order exactly while the simplex analysis resulted in the inter- 
change of levels B and C. This result, together with the small difference 
between unit subtest means for these levels and the closeness of percent 
mastery figures, indicates that there is not a clear-cut prerequisite 
relationship between levels B and C. 

Results such as those summarized in Table 20 reveal a rather 
disappointing situation for the person attempting to verify his hypothesized 
sequence or seeking empirical guidance for generating a better sequence. 
Actually, as far as the IPX math curriculum is concerned, these results 
may only serve to help to identify soim^of the causes of the difficulties 
that pupils were having with the place value sequence. This has been a 
content area that has been of continuing difficulty with respect to questions 
of pupil placement and speed of pupil progression. The hierarchical ordering 
of objectives and units has posed a problem in terms of agreement among 
curriculum writers in their logical analysis of the abilities involved. 
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The results summarized in Table 20 verify the real existence of this 
problem and suggest the need for a rather complete re-analysis of this 
area combined with a reexamination of all tests involved. 

Tables 21, 22, 23, and 24 all reveal situations where there is 
complete, agreement among hypothesized orders, simplex analysis, and 
scalogram analysis. Obviously this is a most satisfying result to the 
curriculum developer attempting to evaluate his curriculum hierarchies. 

A criticism^of scalogram analysis as a tool for substantiating the existence 
of learning hierarchies is that results are ultimately a function of item 
difficulty. Under many conditions the relative difficulty of an item or 
of a test may be independent of its position in a hierarchical learning 
sequence. Since results from a simplex analysis are dependent upon 

intercorrelations and are independent of difficulty, the fact that the 

b 

two analyses agree suggests that the hierarchy that is identified is not 
merely a function of test difficulty. This points to the value of using 
both the scalogram and the simplex analysis in studying curriculum hierar- 
chies. 



Conclusions 

This study has attempted to demonstrate the use of scalogram 
analysis and simplex analysis in the investigation of hierarchical 
relationships among objectives and among units and levels of study in 
an individualized mathematics curriculum. Its goal has been to contri- 
bute to formative evaluation methodology for use in curriculum development. 
Although conclusive answers to questions concerning the most effective 
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teaching sequences may require experimental studies, the procedures used 

V 

here have been found to be helpful in ^ffering suggestions for the revision 

and refinement of such sequences. Thi? study has demonstrated the use- 

. / - 

fulness of both simplex and scalogram analysis for the purpose of assessing 

hypothesized hierarchical relationships among specific behavioral objec- 
tives as well as curriculum units. Areas both in curriculum structure and 
in test structure hav^ been identified which require reexamination and 
modification. The implications of the results of this study are influen- 
cing the current work oii^the revision of the-IPI mathematics curriculum. 

V' 

Some curriculum units have been redefined and certain areas such as 
Numeration and Place Value have been combined in an effort to create a 
more logical ordering of the instructional objectives. The alternative 
structures suggested by these procedures may also be helpful in providing 
guidance for curriculum and testing specialists in the design of future 
curriculums in IPX mathematics. 

/ 

The experience of the investigators in this study has served to 

'N / 

confirm the point that empirical procedures such as scalogram /analysis and 



\ 



simplex analysis are only useful supplements to the careful/ logical analys-is 

that must be involved in the ^original structuring of sequences and 

1 \ 

hierarchies. Certainly these empirical procedures cannot be] used to { 
generate such sequences. Also, the study of curriculum hierarchies 'should 
involve a number of case studies in which individual students are closely 
followed to determine the extent to which specified prerequisite learnings 
do indeed provide 't^ie necessary basis for efficient progression. However, 
in relatively lar^e curriculum projects, such as IPI, where data on pupil 

: i \ 

mastery and progression are regularly collected, both scalogram and simplex 
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^/analysis provide useful procetk for obtaining insights concerning the 
existence of hypothesised hierarchies. Where both procedures can be 

V* / 

applied, the writers have found considerable agreement betvjeen results 

/ • 

\ obtained but have felt that' being able to study agreements ‘and disagree- 

I ‘ 

ments makes the joint employment of both analyses* a worthwhile step. 
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TABLE 1 



\ 



i, 

i 



I 



NUMBER OF OBJECTIVES PER UNIT WHICH ARE 
REPRESENTED ON THE PLACEMENT TESTS 



Number of Objectives Per Area 
Num. PX.V. Add. Sub. Mult. Dlv. Total 



to 

(V 

> 

4J 

O 

QJ iH 
Ti <y 
M > 

o <t> 

o u 
o 



G 

F 

E 

D 

C 



0> B 



4 - 1 

3 2 

2 2 

3 3 

4 2 

3 2 



2 

2 

2 

2 

3 

0 



3 2 



1 3 



2 ’ 3 

2 ' 3 



2 0 



0 0 



3 15 

3 14 

2 13 

3 16 

0 11 

0 5 



Total 



19 12 11 10 



11 



11 74 
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TABLE 2 



COMPARISON OF HYPOTHESIZED AND EMPIRICALLY DERIVED SCALES 
FOR NUMERATION AND PLACE VALUE OBJECTIVES 
EXPANDED NOTATION, INTEGER NUMBERS 
N=74 



„ ( 


\ 

Hypothesised 


1 

1 


Empirical 




Permutation 


Homo aeneous 


Empirical 


Mastery 




Index of 


Grpup 

XX 


i\ Scale / 


Scale 


Percentage 


REP 


Scale Difference 


• / ) 


/ 










Expanded 


PGl 


PGl 


• .0 


'.,:936 


13 


Notation 






i 

17.6 ! 


/ 

f 


PF1 


NF2 






Integer 






/ 






Numbers 


NF2 


PE2 


27.0 








PE2 


PB1 


71.6 








PDA 


PCI 


85.1 








PC5 


PF1 


85.1 








PCI 


PB3 


86.5 








PB3 


PCS 


93.2 








PB1 


PD4 


95.9 








l 


Y TABLE 


3 






COMPARISON OF HYPOTHE 


SIZED AND EMPIRICALLY DERIVED 


SCALES 




FOR NUMERATION AND PLACE VALUE OBJECTIVES 




| 




COUNTING 






L 




N=74 








/ t 

Homogeneous 


Hypothesized 


Empirical 


Empirical 

Mastery 




Permutation 
Index of 


Group 


Scale 


Scale 


Percentage 


RE? 


Scale Difference 

i 


Counting 


ND3 . 


ND3 


60.8 


.946 


1 

)o 



/ 



/ 
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ND2 

NC7 

NC6 

NC5 

NC4 

NB7 



ND2 
NC7 
NC6 
NC5 
■ NC4 
NB7 



64.9 

85.1 
'89.2 

93.2 

97.3 



97.3 



-1 



TABLE '4 

COMPARISON OF HYPOTHESIZED AND EMPIRICALLY DERIVED SCALES 
FOR NUMERATION AND PLACE VALUE OBJECTIVES 
N=74 









Empirical 




Permutation 


Homogeneous 


Hypothesized- 


Empirical 


Mastery 




Index of 


‘t'A'Group 


Scale 


Scale 


Percentage 


REP 


Scale Difference 


Inequality 


PE3 


PE3 


82.4 . 


.936 


0 




NB9 


NB9 


87.8 








NB8 


NB8 


93.2 










l 








Rounding 


NF1 


NF1 


8.1 


.972 


0 


Estimates 


NE3 


NE3 


25.7 






Base 


NG5 


NG5 


2.7 


.982 


0 


Conversion 


NG4 


NG4 


4.1 








NG3 


NG3 


5.4 






Exponential 


NG8 


NG8 


1.4 


1.000 


0 


Form 


PF4 


PF4 


31.1 






Expanded 


PD9 


PD7 


67.6 


.946 


1 


Notation 

Decimal 


PD7 


PD9 


79.7 






Numbers 










l 


Decimal 


NE5 


NE5 


24.3 


.986 


0 'i 


Conversion 














ND5 


ND5 


44.6 
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TABLE 5 



\ 

COMPARISON OF HYPOTHESIZED AND EMPIRICAI . i DERIVED SCALES 





FOR ADDITION AND PLACE VALUE OBJECTIVES 




r\ 


INTEGER 


ADDITION - FACTS AND ALGORITHM 








N=76 














Empii'ical 


Permutation 


Homogeneous 


Hypothesize 


>d Empirical 


Mastery 


Index of 


Group 


Scale 


Scale 


Percentage REP 


Scale Difference 


Integer ^ _ 


' AF1 


AE5 


34.2 .828 


2 


Addition 










Facts end 


AE6 


AF1 


42.1 




Algorithm 


AE4 


AE4 


48.7 






AD8 


AD6 


64.5 






AD6 


AD8 


76.3 






AC3 


AC 3 


80.3 






AC 2 


AC 2 


90.8 






AC1 


AC1 


90.8 








' TABLE 


6 




COMPARISON OF HYPOTHESIZED AND 


EMPIRICALLY DERIVED 


SCALES 




FOR ADDITION AND SUBTRACTION OBJECTIVES 






INTEGER 


SUBTRACTION - 


FACTS AND ALGORITHM 








N=76 













Empirical 




Permutation 


Homogeneous 


Hypothesized 


Empirical 


Mastery 




Index of 


Group 


Scale 


Scale 


Percentage 


REP 


Scale Difference 


Integer 


SE3 


SE3 


19.7 


.936 


i 


Subtraction < 










Facts and 


y . SD5 


SD5 


39.5 






Algorithm 


k » 

SD4 


SD4 


52.6 








SC3 


SCI 


81.6 








SCI 


SC3 


93.4 
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TABLE 7* 



COMPARISON OF HYPOTHESIZED AND EMPIRICALLY DERIVED SCALES 
() PGR ADDITION AND SUBTRACTION OBJECTIVES 

' i • n=76 









Empirical 


Permutation 


Homogeneous 


Hypothesized 


Empirical 


Mastery 


Index of 


Group 


Scale \ 

\ 

v 


Scale 


Percentage REP 


Scale Difference 



Addition 

Subtraction 


SFl 


SFl 


Decimal 

Numbers 


AF2 


SE2 




SE2 


AF2 


Addition 

Subtraction 


SG2 • 


SGI 


Negative 

Numbers 


SGI 


AGl 




AGl 


SG2 


Addition 

Subtraction 


SG3 


AG 3 


Exponential 

Numbers 


AG 3 


SG3 



18.4 .912 1 

31.6 

35.5 

0 .974 2 

7.9 

19.7 

14.5 .960 1 

23.7 
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TABLE 8 



COMPARISON OF HYPOTHESIZED AND EMPIRICALLY DERIVED SCALES 
FOR MULTIPLICATION AND DIVISION OBJECTIVES 
MULTIPLICATION ALGORITHM - INTEGER NUMBERS 

N=74 









Empirical 




Permutation 


Homogeneous Hyp 


othesized 


Empirical 


Mastery 




Index of 


Group 


Scale 


Scale 


Percentage 


REP 


Scale Difference 


Multiplication 


MFIO 


MF5 


10.8 


.946 


i 


Algorithm 

Integer 


MF5 


MFIO 


12.2 






Numbers 


MEll 


MEll 


29.7 








MElO 


MElO 


29.7 








ME7 


ME7 


35.1 







TABLE 9 

COMPARISON OF HYPOTHESIZED AND EMPIRICALLY DERIVED SCALES 
FOR MULT I P L I CAT I ON AND DIVISION OBJECTIVES 
DIVISION - CONCEPT AND FACTS 
N=74 



Homogeneous 

Group 


Hypothesized 

Scale 


Empirical 

Scale 


Empirical 

Mastery 

Percentage 


REP 


Permutation 
Index of 

Scale Difference 


Division 


DE7 


DE7 


31.1 


.878 


2 


Concept and 
Facts 


DD7 


DD5 


56.8 








DD5 


DD2 


64.9 








DD2 


DD7 


77.0 
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TABLE 10 



COMPARISON OF HYPOTHESIZED AND EMPIRICALLY DERIVED SCALES 
FOR MULTIPLICATION AND DIVISION OBJECTIVES 

N=74 



Homogeneous Hypothesized Empirical 
Group Scale Scale 



Empirical 


Permutation 


Mastery 


Index of 


Percentage RHP 


Scale Difference 



Multiplication MD3 MD8 



Concept and 


MD4 


MD3 


Pacta 


MD3 


MD4 



Multiplication 


MG6 , 


MG6 


Algorithm 


MG5 


MG5 


Decimal 






Numbers 

/ 


. MF9 


■ MF9 


Division 


DF6 


DF6 


Algorithm 


DF4 


DF4 


Integer 






Numbers — 


DE5 


DE5 



Division 


DG5 


DG5 


Algorithm 


DG4 


DG4 


Decimal 






Numbers 


DF7 


DF7 



77.0 .820 1 

81.1 
87.8 

2.7 1.000 0 

2.7 

2.7 

2.7 .982 0 

4.1 
13.5 

.0 1.000 0 

.0 

1.4 
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CORRELATIONS AMONG IPI MATHEMATICS AREAS ON 
LEVEL D PLACEMENT TESTS (INPUT MATRIX) 
N°235 



Content 

Area 


Numeration 


Place 

Value 


Addition 


Subtraction 


Multiplication 


Division 


Nura . 


1.00 


0.36 


0.38 


0.48 


0.54 


0.50 


P.V. 


0.36 


1.00 


0.12 


0.32 


0.27 


0.28 


Add. 


0.38 


0.12 


1.00 


0.59 


0.69 


0.61 


Sub . 


0.48 


0.32 


0.59 


1.00 


0.69 


0.68 


Mult. 


0.54 


0.27 


0.69 


0.69 


1,00 


0.77 


Div. ' 


0.50 


0.28 


0.61 


0.68 • 


0.77 


1.00 



TABLE 12 

CORRELATIONS AMONG IPX H&XHEKA3XCS AREAS ON LEVEL D PLACEMENT TESTS 
(OPTIMAL ORDERING FROM SIMPLEX ANALYSIS) 

N»235 



Content 

Area 


Addition Multiplication 


Division 


Subtraction 


Place 

Numeration Value 


Add. 


1.00 


0.69 


0.61 


0.59 


0.38 


0.12 


Mult. 


0.69 


1.00 


0.77 


0.69 


0.54 


0.27 


Div. 


0.61 " 


0.77 


1.00 


0.68 


0.50 


0.28 


Sub . 


0.59 


0.69 


0.68 


1.00 


0.48 


0.32 


Num. 


0.38 


0.54 


0.50 


0.48 


1.00 


0.36 


P.V. 


0.12 


0.27 


0.28 


0.32 


0.36 


1.00 
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TABLE 13 



CORRELATIONS AMONG IP I MATHEMATICS AREAS ON LEVEL E 
PLACEMENT TESTS (INPUT MATRIX) 

N*»89 



Content 

Aires 


Numeration 


Place 

Value 


Addition 


Subtraction 


Multiplication 


Division 


Nisn. 


1.00 


0.34 


0.05 


0.06 


0.21 


0.14 


P.V. 


0.34 


1.00 


0.13 


0.07 


0.30 

\ 


0.15 


Add. 


0.05 


0.13 


1.00 


0.38 


0.27 


0.29 


Sub. 


0.06 


0.07 


0.38 


1.00 


0.38 


0.21 


Mult. 


0.21 


0.30 


0.27 


0.38 


1.00 


0.62 


Div. 


0.14 


0.15 


0.29 


0.21 


0.62 


1.00 





TABLE 14 

CORRELATIONS AMONG IPI MATHEMATICS AREAS ON LEVEL E 

placement: TESTS (optimal ordering from simplex 
ANALYSIS) N-89 




Content 

Area 


Subtraction 


Addition 


Division 


Multiplication 


Place 

Value 


Numeration 


Sub. 


1.00 


0.38 


0.21 




0.38 


0.07 


0.06 


Add. 


0.38 


1.00 


0.29 




0.27 


0.13 


0.05 


Div. 


0.21 


0.29 


1.00 




0.62 


0.15 


0.14 


Mult. 


0.38 


0.27 


0.62 




1.00 


0.30 


0.21 


P.V. 


0.07 


0.13 


0.15 




0.30 


1.00 


0.34 


Num. 


0.06 


0.05 


0.14 




0.21 


0.34 


1.00 
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TABLE 15 



CORRELATIONS AMONG IPI MATHEMATICS LEVELS ON NUMERATION 
PLACEMENT TESTS (INPUT MATRIX) 

N-74 



Content 

Level 


B 


C 


D 


.E 


F 


G 


B 


1.00 


0.63 


0.50 


0.36 


0.24 


0.09 


C 


0.63 


1.00 


0.45 


0.36 


0.24 


0.09 


D 


0.50 


0.45 


1.00 


0.67 


0.54 


0.22 


E 


0.36 


0.36 


0.67 


1.00 


0.64 


0.26 


F 


0.24 


0.24 


0.54 


0.64 


1.00 


0.50 


G 


0.09 


0.09 


0.22 


0.26 


0.50 


1.00 



TABLE 16 

CORRELATIONS AMONG IPI MATHEMATICS LEVELS ON NUMERATION PLACEMENT 
TESTS (OPTIMAL ORDERING FROM SIMPLEX ANALYSIS) 

N**74 



Content 



Level 


C 


B 


C 


1.00 


0.63 


B 


’ 0.63 


1.00 


D 


0.45 


0.50 


E 


0.36 


0.36 


F 


0.24 


0.24 


G 


0.09 


0.09 



ERIC 



D 


E 


F 


G 


0.43 


0.36 


0.24 


0.09 


0.50 


0.36 


0.24 


0.09 


1.00 


0.67 


0.54 


0.22 


0.67 


1.00 


0.64 


0.26 


0.54 


0.64 


1.00 


0.50 


0.22 


0.26 


0.50 


1.00 
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TABLE 17 



CORRELATIONS AMONG IPI MATHEMATICS LEVELS ON PLACE VALUE 
PLACEMENT TESTS (INPUT MATRIX) 

N»74 



Content 

Level 


B 


c 


D 


E 


F 


G 


B 


1.00 


0.29 


0.22 


0.45 


0.37 


0.54 


C 


0.29 


1.00 


0.43 


0.35 


0.41 


0.28 


D 


0.22 


0.43 


1.00 


0.27 


0.34 


0.17 


E 


0.45 


0.35 


0.27 


1.00 


0.65 


0.62 


F 


0.37 


0.41 


0.34 


0.65 


1.00 


0.57 


G 


0.54 


0.28 


0.17 


0.62 


0.57 


1.00 



TABLE 18 



CORRELATIONS AMONG IPI MATHEMATICS LEVELS ON PLACE VALUE PLACEMENT 
TESTS (OPTIMAL ORDERING FROM SIMPLEX ANALYSIS) 

N“74 



Content 



Level 


D 


c 


F 


E 


G 


w 


D 


1.00 


0.43 


0.34 


0.27 


0.17 ' 


0.22 


C 


0.43 


1.00 


0.41 


0.35 


0.28 


0.29/ 


F 


0.34 


0.41 


1.00 


0.65 


0.57 . 


0 . 37' 


E 


0.27 


0.35 


0.65 


1.00 


0.62 


0.45 


G 


0.17 


0.28 


0.57 


0.62 


1.00 


0.-54 


B 


0.22 


0.29 


0.37 


0.45 


0.54 


1.00 
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A COMPARISON OF THE COMPLICITY ORDER OF IPI MATHEMATICS LEV!.. 
CONTENT AREA OF NUMERATION OBTAINED FROM SIMPLEX AND 
SCALOGRAM ANALYSES 

N«74 / 



'll THE 



Simplex Analysis 



Unit Sub teat 
Means 



3.2 

15.7 

38.4 

58.1 
94.6 

91.2 



Complexity 

Order 



G 

F 

E 

D 

B 

C 



Scalogram Analysis 



Complexity 

Order 



G 

F 

E 

D 

C 

B 



Percent 

Mastery 



1.4 

5.4 
25.7 
47.3 
89.2 
91.9 



TABLE 20 

/ 

A COMPARISON OF THE COMPLEXITY ORDER OF IPI MATHEMATICS LEVELS IN THE 
CONTENT AREA OF PLACE VALUE OBTAINED FROM SIMPLEX AND 
SCALOGRAM ANALYSES 
N*»74 



. Simplex Analysis 



70.5 

93.5 



ERIC 



F 

C 

D 



Scalogram Analysis 



D 

B 

C 




79.7 

81.1 



93.2 



TABLE 21 



A COMPARISON OF THE COMPLEXITY ORDER OF IPI MATHEMATICS LEVELS IN THE 
CONTENT AREA OF ADDITION OBTAINED FROM SIMPLEX AND 
SCALOGRAM ANALYSES 
N-76 



Simplex Analysis 


Scalogram Analysis 


Unit Sub test 


Complexity 


Complexity 


Percent 


Means 


Order 


Order 


Mastery 


15.5 


G 


G 


9.2 


49.0 


F 


F 


46.1 


57.4 


E 


E 


52.6 


77.1 


D 


D 


69.7 


94.3 


C 


C 


96.1 







TABLE 22 






A COMPARISON OF 
CONTENT 


THE COMPLEXITY ORDER OF IPI MATHEMATICS LEVELS 
area OF SUBTRACTION OBTAINED FROM SIMPLEX AND 
SCALOGRAM ANALYSES 
N“76 


; IN THE 


Simplex Analysis 




Scalogram Analysis 


Unit Sub test 
Means 


Complexity 

Order 




Complexity 

Order 


Percent 

Mastery 


15.3 


G 




G 


1.3 


39.2 


F 




F 


28.9 


39.2 


E 




E 


30.3 


54.0 


D 




D 


47.4 


95.5 


C 




C 


93.4 
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TABLE 23 



A COMPARISON OF THE COMPLEXITY ORDER OF IPI MATHEMATICS LEVELS IN THE 
CONTENT AREA OF MULTIPLICATION OBTAINED FROM SIMPLEX AND 
SCALOGRAM /ANALYSES 
N“74 



Simplex Analysis 




Scalograiu Analysis 


Unit Subteat 
Means 


Complexity 

Order 




Complexity 

Order 


Percent 

Mastery 


3.0 


G 




G 


2.7 

« 


10.0 


F 




F 


2.7 


31.4 


E 




E 


21.6 


92.8 


D 




D 


94.6 







TABLE 24 






A COMPARISON OF THE COMPLEXITY ORDER OF IPI 
CONTENT AREA OF DIVISION OBTAINED 
SCALOGRAM ANALYSES 
N**74 


MATHEMATICS LEVELS IN THE 
FROM SIMPLEX AND 


Simplex Analysis 




Scalogram Analysis 


Unit Subtest 
Means 


Complexity 
* Order 




Complexity 

Order 


Percent 

Mastery 


0.8 


G 




G 


0.0 


4.6 


F 




F 


0.0 


29.0 


E 




E 


21.6 


80.7 


D 




D 


71.6 
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